Development and Implementation of Digital Diagnostic Algorithms for Neonatal Units in Zimbabwe and Malawi: Development and Usability Study.

BACKGROUND: Despite an increase in hospital-based deliveries, neonatal mortality remains high in low-resource settings. Due to limited laboratory diagnostics, there is significant reliance on clinical findings to inform diagnoses. Accurate, evidence-based identification and management of neonatal conditions could improve outcomes by standardizing care. This could be achieved through digital clinical decision support (CDS) tools. Neotree is a digital, quality improvement platform that incorporates CDS, aiming to improve neonatal care in low-resource health care facilities. Before this study, first-phase CDS development included developing and implementing neonatal resuscitation algorithms, creating initial versions of CDS to address a range of neonatal conditions, and a Delphi study to review key algorithms. OBJECTIVE: This second-phase study aims to codevelop and implement neonatal digital CDS algorithms in Malawi and Zimbabwe. METHODS: Overall, 11 diagnosis-specific web-based workshops with Zimbabwean, Malawian, and UK neonatal experts were conducted (August 2021 to April 2022) encompassing the following: (1) review of available evidence, (2) review of country-specific guidelines (Essential Medicines List and Standard Treatment Guidelinesfor Zimbabwe and Care of the Infant and Newborn, Malawi), and (3) identification of uncertainties within the literature for future studies. After agreement of clinical content, the algorithms were programmed into a test script, tested with the respective hospital's health care professionals (HCPs), and refined according to their feedback. Once finalized, the algorithms were programmed into the Neotree software and implemented at the tertiary-level implementation sites: Sally Mugabe Central Hospital in Zimbabwe and Kamuzu Central Hospital in Malawi, in December 2021 and May 2022, respectively. In Zimbabwe, usability was evaluated through 2 usability workshops and usability questionnaires: Post-Study System Usability Questionnaire (PSSUQ) and System Usability Scale (SUS). RESULTS: Overall, 11 evidence-based diagnostic and management algorithms were tailored to local resource availability. These refined algorithms were then integrated into Neotree. Where national management guidelines differed, country-specific guidelines were created. In total, 9 HCPs attended the usability workshops and completed the SUS, among whom 8 (89%) completed the PSSUQ. Both usability scores (SUS mean score 75.8 out of 100 [higher score is better]; PSSUQ overall score 2.28 out of 7 [lower score is better]) demonstrated high usability of the CDS function but highlighted issues around technical complexity, which continue to be addressed iteratively. CONCLUSIONS: This study describes the successful development and implementation of the only known neonatal CDS system, incorporated within a bedside data capture system with the ability to deliver up-to-date management guidelines, tailored to local resource availability. This study highlighted the importance of collaborative participatory design. Further implementation evaluation is planned to guide and inform the development of health system and program strategies to support newborn HCPs, with the ultimate goal of reducing preventable neonatal morbidity and mortality in low-resource settings.

Development and implementation experience of a learning healthcare system for facility based newborn care in low resource settings: The Neotree.

Introduction: Improving peri- and postnatal facility-based care in low-resource settings (LRS) could save over 6000 babies' lives per day. Most of the annual 2.4 million neonatal deaths and 2 million stillbirths occur in healthcare facilities in LRS and are preventable through the implementation of cost-effective, simple, evidence-based interventions. However, their implementation is challenging in healthcare systems where one in four babies admitted to neonatal units die. In high-resource settings healthcare systems strengthening is increasingly delivered via learning healthcare systems to optimise care quality, but this approach is rare in LRS. Methods: Since 2014 we have worked in Bangladesh, Malawi, Zimbabwe, and the UK to co-develop and pilot the Neotree system: an android application with accompanying data visualisation, linkage, and export. Its low-cost hardware and state-of-the-art software are used to support healthcare professionals to improve postnatal care at the bedside and to provide insights into population health trends. Here we summarise the formative conceptualisation, development, and preliminary implementation experience of the Neotree. Results: Data thus far from ~18 000 babies, 400 healthcare professionals in four hospitals (two in Zimbabwe, two in Malawi) show high acceptability, feasibility, usability, and improvements in healthcare professionals' ability to deliver newborn care. The data also highlight gaps in knowledge in newborn care and quality improvement. Implementation has been resilient and informative during external crises, for example, coronavirus disease 2019 (COVID-19) pandemic. We have demonstrated evidence of improvements in clinical care and use of data for Quality Improvement (QI) projects. Conclusion: Human-centred digital development of a QI system for newborn care has demonstrated the potential of a sustainable learning healthcare system to improve newborn care and outcomes in LRS. Pilot implementation evaluation is ongoing in three of the four aforementioned hospitals (two in Zimbabwe and one in Malawi) and a larger scale clinical cost effectiveness trial is planned.

Protocol for an intervention development and pilot implementation evaluation study of an e-health solution to improve newborn care quality and survival in two low-resource settings, Malawi and Zimbabwe: Neotree.

INTRODUCTION: Every year 2.4 million deaths occur worldwide in babies younger than 28 days. Approximately 70% of these deaths occur in low-resource settings because of failure to implement evidence-based interventions. Digital health technologies may offer an implementation solution. Since 2014, we have worked in Bangladesh, Malawi, Zimbabwe and the UK to develop and pilot Neotree: an android app with accompanying data visualisation, linkage and export. Its low-cost hardware and state-of-the-art software are used to improve bedside postnatal care and to provide insights into population health trends, to impact wider policy and practice. METHODS AND ANALYSIS: This is a mixed methods (1) intervention codevelopment and optimisation and (2) pilot implementation evaluation (including economic evaluation) study. Neotree will be implemented in two hospitals in Zimbabwe, and one in Malawi. Over the 2-year study period clinical and demographic newborn data will be collected via Neotree, in addition to behavioural science informed qualitative and quantitative implementation evaluation and measures of cost, newborn care quality and usability. Neotree clinical decision support algorithms will be optimised according to best available evidence and clinical validation studies. ETHICS AND DISSEMINATION: This is a Wellcome Trust funded project (215742_Z_19_Z). Research ethics approvals have been obtained: Malawi College of Medicine Research and Ethics Committee (P.01/20/2909; P.02/19/2613); UCL (17123/001, 6681/001, 5019/004); Medical Research Council Zimbabwe (MRCZ/A/2570), BRTI and JREC institutional review boards (AP155/2020; JREC/327/19), Sally Mugabe Hospital Ethics Committee (071119/64; 250418/48). Results will be disseminated via academic publications and public and policy engagement activities. In this study, the care for an estimated 15 000 babies across three sites will be impacted. TRIAL REGISTRATION NUMBER: NCT0512707; Pre-results.

Indirect impacts of the COVID-19 pandemic at two tertiary neonatal units in Zimbabwe and Malawi: an interrupted time series analysis.

OBJECTIVES: To examine indirect impacts of the COVID-19 pandemic on neonatal care in low-income and middle-income countries. DESIGN: Interrupted time series analysis. SETTING: Two tertiary neonatal units in Harare, Zimbabwe and Lilongwe, Malawi. PARTICIPANTS: We included a total of 6800 neonates who were admitted to either neonatal unit from 1 June 2019 to 25 September 2020 (Zimbabwe: 3450; Malawi: 3350). We applied no specific exclusion criteria. INTERVENTIONS: The first cases of COVID-19 in each country (Zimbabwe: 20 March 2020; Malawi: 3 April 2020). PRIMARY OUTCOME MEASURES: Changes in the number of admissions, gestational age and birth weight, source of admission referrals, prevalence of neonatal encephalopathy, and overall mortality before and after the first cases of COVID-19. RESULTS: Admission numbers in Zimbabwe did not initially change after the first case of COVID-19 but fell by 48% during a nurses' strike (relative risk (RR) 0.52, 95% CI 0.41 to 0.66, p<0.001). In Malawi, admissions dropped by 42% soon after the first case of COVID-19 (RR 0.58, 95% CI 0.48 to 0.70, p<0.001). In Malawi, gestational age and birth weight decreased slightly by around 1 week (beta -1.4, 95% CI -1.62 to -0.65, p<0.001) and 300 g (beta -299.9, 95% CI -412.3 to -187.5, p<0.001) and outside referrals dropped by 28% (RR 0.72, 95% CI 0.61 to 0.85, p<0.001). No changes in these outcomes were found in Zimbabwe and no significant changes in the prevalence of neonatal encephalopathy or mortality were found at either site (p>0.05). CONCLUSIONS: The indirect impacts of COVID-19 are context-specific. While our study provides vital evidence to inform health providers and policy-makers, national data are required to ascertain the true impacts of the pandemic on newborn health.

Usability-Focused Development and Usage of NeoTree-Beta, an App for Newborn Care in a Low-Resource Neonatal Unit, Malawi.

Background: Neonatal mortality is high in low-resource settings. NeoTree is a digital intervention for neonatal healthcare professionals (HCPs) aiming to achieve data-driven quality improvement and improved neonatal survival in low-resource hospitals. Optimising usability with end-users could help digital health interventions succeed beyond pilot stages in low-resource settings. Usability is the quality of a user's experience when interacting with an intervention, encompassing their effectiveness, efficiency, and overall satisfaction. Objective: To evaluate the usability and usage of NeoTree beta-app and conduct Agile usability-focused intervention development. Method: A real-world pilot of NeoTree beta-app was conducted over 6 months at Kamuzu Central Hospital neonatal unit, Malawi. Prior to deployment, think-aloud interviews were conducted to guide nurses through the app whilst voicing their thoughts aloud (n = 6). System Usability Scale (SUS) scores were collected before the implementation of NeoTree into usual clinical care and 6 months after implementation (n = 8 and 8). During the pilot, real-world user-feedback and user-data were gathered. Feedback notes were subjected to thematic analysis within an Agile "product backlog." For usage, number of users, user-cadre, proportion of admissions/outcomes recorded digitally, and median app-completion times were calculated. Results: Twelve overarching usability themes generated 57 app adjustments, 39 (68%) from think aloud analysis and 18 (32%) from the real-world testing. A total of 21 usability themes/issues with corresponding app features were produced and added to the app. Six themes relating to data collection included exhaustiveness of data schema, prevention of errors, ease of progression, efficiency of data entry using shortcuts, navigation of user interface (UI), and relevancy of content. Six themes relating to the clinical care included cohesion with ward process, embedded education, locally coherent language, adaptability of user-interface to available resources, and printout design to facilitate handover. SUS scores were above average (88.1 and 89.4 at 1 and 6 months, respectively). Ninety-three different HCPs of 5 cadres, recorded 1,323 admissions and 1,197 outcomes over 6 months. NeoTree achieved 100% digital coverage of sick neonates admitted. Median completion times were 16 and 8 min for admissions and outcomes, respectively. Conclusions: This study demonstrates optimisation of a digital health app in a low-resource setting and could inform other similar usability studies apps in similar settings.

Software development process of Neotree - a data capture and decision support system to improve newborn healthcare in low-resource settings.

The global priority of improving neonatal survival could be tackled through the universal implementation of cost-effective maternal and newborn health interventions. Despite 90% of neonatal deaths occurring in low-resource settings, very few evidence-based digital health interventions exist to assist healthcare professionals in clinical decision-making in these settings. To bridge this gap, Neotree was co-developed through an iterative, user-centered design approach in collaboration with healthcare professionals in the UK, Bangladesh, Malawi, and Zimbabwe. It addresses a broad range of neonatal clinical diagnoses and healthcare indicators as opposed to being limited to specific conditions and follows national and international guidelines for newborn care. This digital health intervention includes a mobile application (app) which is designed to be used by healthcare professionals at the bedside. The app enables real-time data capture and provides education in newborn care and clinical decision support via integrated clinical management algorithms. Comprehensive routine patient data are prospectively collected regarding each newborn, as well as maternal data and blood test results, which are used to inform clinical decision making at the bedside. Data dashboards provide healthcare professionals and hospital management a near real-time overview of patient statistics that can be used for healthcare quality improvement purposes. To enable this workflow, the Neotree web editor allows fine-grained customization of the mobile app. The data pipeline manages data flow from the app to secure databases and then to the dashboard. Implemented in three hospitals in two countries so far, Neotree has captured routine data and supported the care of over 21,000 babies and has been used by over 450 healthcare professionals. All code and documentation are open source, allowing adoption and adaptation by clinicians, researchers, and developers.

Refining clinical algorithms for a neonatal digital platform for low-income countries: a modified Delphi technique.

OBJECTIVES: To determine whether a panel of neonatal experts could address evidence gaps in local and international neonatal guidelines by reaching a consensus on four clinical decision algorithms for a neonatal digital platform (NeoTree). DESIGN: Two-round, modified Delphi technique. SETTING AND PARTICIPANTS: Participants were neonatal experts from high-income and low-income countries (LICs). METHODS: This was a consensus-generating study. In round 1, experts rated items for four clinical algorithms (neonatal sepsis, hypoxic ischaemic encephalopathy, respiratory distress of the newborn, hypothermia) and justified their responses. Items meeting consensus for inclusion (≥80% agreement) were incorporated into the algorithms. Items not meeting consensus were either excluded, included following revisions or included if they contained core elements of evidence-based guidelines. In round 2, experts rated items from round 1 that did not reach consensus. RESULTS: Fourteen experts participated in round 1, 10 in round 2. Nine were from high-income countries, five from LICs. Experts included physicians and nurse practitioners with an average neonatal experience of 20 years, 12 in LICs. After two rounds, a consensus was reached on 43 of 84 items (52%). Per experts' recommendations, items in line with local and WHO guidelines yet not meeting consensus were still included to encourage consistency for front-line healthcare workers. As a result, the final algorithms included 53 items (62%). CONCLUSION: Four algorithms in a neonatal digital platform were reviewed and refined by consensus expert opinion. Revisions to NeoTree will be made in response to these findings. Next steps include clinical validation of the algorithms.

Electronic application to improve management of infections in low-income neonatal units: pilot implementation of the NeoTree beta app in a public sector hospital in Zimbabwe.

There are 2. 4 million annual neonatal deaths worldwide. Simple, evidence-based interventions such as temperature control could prevent approximately two-thirds of these deaths. However, key problems in implementing these interventions are a lack of newborn-trained healthcare workers and a lack of data collection systems. NeoTree is a digital platform aiming to improve newborn care in low-resource settings through real-time data capture and feedback alongside education and data linkage. This project demonstrates proof of concept of the NeoTree as a real-time data capture tool replacing handwritten clinical paper notes over a 9-month period in a tertiary neonatal unit at Harare Central Hospital, Zimbabwe. We aimed to deliver robust data for monthly mortality and morbidity meetings and to improve turnaround time for blood culture results among other quality improvement indicators. There were 3222 admissions and discharges entered using the NeoTree software with 41 junior doctors and 9 laboratory staff trained over the 9-month period. The NeoTree app was fully integrated into the department for all admission and discharge documentation and the monthly presentations became routine, informing local practice. An essential factor for this success was local buy-in and ownership at each stage of the project development, as was monthly data analysis and presentations allowing us to rapidly troubleshoot emerging issues. However, the laboratory arm of the project was negatively affected by nationwide economic upheaval. Our successes and challenges piloting this digital tool have provided key insights for effective future roll-out in Zimbabwe and other low-income healthcare settings.

Admissions to a Low-Resource Neonatal Unit in Malawi Using a Mobile App and Dashboard: A 1-Year Digital Perinatal Outcome Audit.

Introduction: Understanding the extent and cause of high neonatal deaths rates in Sub-Saharan Africa is a challenge, especially in the presence of poor-quality and inaccurate data. The NeoTree digital data capture and quality improvement system has been live at Kamuzu Central Hospital, Neonatal Unit, Malawi, since April 2019. Objective: To describe patterns of admissions and outcomes in babies admitted to a Malawian neonatal unit over a 1-year period via a prototype data dashboard. Methods: Data were collected prospectively at the point of care, using the NeoTree app, which includes digital admission and outcome forms containing embedded clinical decision and management support and education in newborn care according to evidence-based guidelines. Data were exported and visualised using Microsoft Power BI. Descriptive and inferential analysis statistics were executed using R. Results: Data collected via NeoTree were 100% for all mandatory fields and, on average, 96% complete across all fields. Coverage of admissions, discharges, and deaths was 97, 99, and 91%, respectively, when compared with the ward logbook. A total of 2,732 neonates were admitted and 2,413 (88.3%) had an electronic outcome recorded: 1,899 (78.7%) were discharged alive, 12 (0.5%) were referred to another hospital, 10 (0.4%) absconded, and 492 (20%) babies died. The overall case fatality rate (CFR) was 204/1,000 admissions. Babies who were premature, low birth weight, out born, or hypothermic on admission, and had significantly higher CFR. Lead causes of death were prematurity with respiratory distress (n = 252, 51%), neonatal sepsis (n = 116, 23%), and neonatal encephalopathy (n = 80, 16%). The most common perceived modifiable factors in death were inadequate monitoring of vital signs and suboptimal management of sepsis. Two hundred and two (8.1%) neonates were HIV exposed, of whom a third [59 (29.2%)] did not receive prophylactic nevirapine, hence vulnerable to vertical infection. Conclusion: A digital data capture and quality improvement system was successfully deployed in a low resource neonatal unit with high (1 in 5) mortality rates providing and visualising reliable, timely, and complete data describing patterns, risk factors, and modifiable causes of newborn mortality. Key targets for quality improvement were identified. Future research will explore the impact of the NeoTree on quality of care and newborn survival.

Admissions to a Low-Resource Neonatal Unit in Malawi Using a Mobile App: Digital Perinatal Outcome Audit.

BACKGROUND: Mobile health (mHealth) is showing increasing potential to address health outcomes in underresourced settings as smartphone coverage increases. The NeoTree is an mHealth app codeveloped in Malawi to improve the quality of newborn care at the point of admission to neonatal units. When collecting vital demographic and clinical data, this interactive platform provides clinical decision support and training for the end users (health care professionals [HCPs]), according to evidence-based national and international guidelines. OBJECTIVE: This study aims to examine 1 month's data collected using NeoTree in an outcome audit of babies admitted to a district-level neonatal nursery in Malawi and to demonstrate proof of concept of digital outcome audit data in this setting. METHODS: Using a phased approach over 1 month (November 21-December 19, 2016), frontline HCPs were trained and supported to use NeoTree to admit newborns. Discharge data were collected by the research team using a discharge form within NeoTree, called NeoDischarge. We conducted a descriptive analysis of the exported pseudoanonymized data and presented it to the newborn care department as a digital outcome audit. RESULTS: Of 191 total admissions, 134 (70.2%) admissions were completed using NeoTree, and 129 (67.5%) were exported and analyzed. Of 121 patients for whom outcome data were available, 102 (84.3%) were discharged alive. The overall case fatality rate was 93 per 1000 admitted babies. Prematurity with respiratory distress syndrome, birth asphyxia, and neonatal sepsis contributed to 25% (3/12), 58% (7/12), and 8% (1/12) of deaths, respectively. Data were more than 90% complete for all fields. Deaths may have been underreported because of phased implementation and some families of babies with imminent deaths self-discharging home. Detailed characterization of the data enabled departmental discussion of modifiable factors for quality improvement, for example, improved thermoregulation of infants. CONCLUSIONS: This digital outcome audit demonstrates that data can be captured digitally at the bedside by HCPs in underresourced newborn facilities, and these data can contribute to a meaningful review of the quality of care, outcomes, and potential modifiable factors. Coverage may be improved during future implementation by streamlining the admission process to be solely via digital format. Our results present a new methodology for newborn audits in low-resource settings and are a proof of concept for a novel newborn data system in these settings.

The NeoTree application: developing an integrated mHealth solution to improve quality of newborn care and survival in a district hospital in Malawi.

More than two-thirds of newborn lives could be saved worldwide if evidence-based interventions were successfully implemented. We developed the NeoTree application to improve quality of newborn care in resource-poor countries. The NeoTree is a fully integrated digital health intervention that combines immediate data capture, entered by healthcare workers (HCW) on admission, while simultaneously providing them with evidence-based clinical decision support and newborn care education. We conducted a mixed-methods intervention development study, codeveloping and testing the NeoTree prototype with HCWs in a district hospital in Malawi. Focus groups explored the acceptability and feasibility of digital health solutions before and after implementation of the NeoTree in the clinical setting. One-to-one theoretical usability workshops and a 1-month clinical usability study informed iterative changes, gathered process and clinical data, System Usability Scale (SUS) and perceived improvements in quality of care. HCWs perceived the NeoTree to be acceptable and feasible. Mean SUS before and after the clinical usability study were high at 80.4 and 86.1, respectively (above average is >68). HCWs reported high-perceived improvements in quality of newborn care after using the NeoTree on the ward. They described improved confidence in clinical decision-making, clinical skills, critical thinking and standardisation of care. Identified factors for successful implementation included a technical support worker. Coproduction, mixed-methods approaches and user-focused iterative development were key to the development of the NeoTree prototype, which was shown to be an agile, acceptable, feasible and highly usable tool with the potential to improve the quality of newborn care in resource-poor settings.

Evaluation of 'TRY': an algorithm for neonatal continuous positive airways pressure in low-income settings.

BACKGROUND: Non-invasive respiratory support using bubble continuous positive airway pressure (bCPAP) is useful in treating babies with respiratory distress syndrome. Despite its proven clinical and cost-effectiveness, implementation is hampered by the inappropriate administration of bCPAP in low-resource settings. A clinical algorithm-'TRY' (based on Tone: good; Respiratory distress; Yes, heart rate above 100 beats/min)-has been developed to correctly identify which newborns would benefit most from bCPAP in a teaching hospital in Malawi. OBJECTIVE: To evaluate the reliability, sensitivity and specificity of TRY when employed by nurses in a Malawian district hospital. METHODS: Nursing staff in a Malawian district hospital baby unit were asked, over a 2-month period, to complete TRY assessments for every newly admitted baby with the following inclusion criteria: clinical evidence of respiratory distress and/or birth weight less than 1.3 kg. A visiting paediatrician, blinded to nurses' assessments, concurrently assessed each baby, providing both a TRY assessment and a clinical decision regarding the need for CPAP administration. Inter-rater reliability was calculated comparing nursing and paediatrician TRY assessment outcomes. Sensitivity and specificity were estimated comparing nurse TRY assessments against the paediatrician's clinical decision. RESULTS: Two hundred and eighty-seven infants were admitted during the study period; 145 (51%) of these met the inclusion criteria, and of these 57 (39%) received joint assessments. The inter-rater reliability was high (kappa 0.822). Sensitivity and specificity were 92% and 96%, respectively. CONCLUSIONS: District hospital nurses, using the TRY-CPAP algorithm, reliably identified babies that might benefit from bCPAP and thus improved its effective implementation.

Preschool children who are frequent attenders in emergency departments: an observational study of associated demographics and clinical characteristics.

BACKGROUND: Unscheduled visits to emergency departments (ED) have increased in the UK in recent years. Children who are repeat attenders are relatively understudied. AIMS: To describe the sociodemographic and clinical characteristics of preschoolers who attend ED a large District General Hospital. METHOD/STUDY DESIGN: Observational study analysing routinely collected ED operational data. Children attending four or more visits per year were considered as 'frequent attenders'. Poisson regression was used with demographic details (age, sex, ethnicity, sociodemographic status) to predict number of attendances seen in the year. We further analysed detailed diagnostic characteristics of a random sample of 10% of attendees. MAIN FINDINGS: 10 169 patients visited in the 12-month period with 16 603 attendances. 655 individuals attended on 3335 occasions. 6.4% of this population accounted for 20.1% of total visits. In the 10% sample, there were 304 attendances, and 69 (23%) had an underlying chronic long-standing illness (CLSI). This group were 2.4 times more likely to be admitted as inpatients compared with those without such conditions, median length of stay of 6.2 hours versus 2.5 hours (p=NS). CONCLUSIONS: Frequent ED attenders fall broadly into two distinct clinical groups: those who habitually return with self-limiting conditions and those with or without exacerbation of underlying CLSI. Both groups may be amenable to both additional nursing and other forms of community support to enhance self-care and continuity of care. Further research is required to increase our understanding of specific individual family and health system factors that predict repeat attendance in this age group.